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ABSTRACT 

This paper discusses a st udy that sought to des ign an 
instrument for assessing the nonverbal ability of foreign language 
learners. The subjects were 28 educated Japanese non-native speakers 
(NNS) of English and 2C educated North American native speakers (NS) 
of English. Materials used were the institutional Test of English as 
a Foreign Language (TOEFL) for linguistic proficiency, the SPEAK test 
for oral proficiency, and a series of four role plays for the 
collection of nonverbal data. Proficiency tests were administered to 
NNS subjects paired according to the results of the TOEFL. NNS and NS 
subjects performed the three plays in pairs and the fourth with the 
researcher's NS assistants. Role plays were transcribed and coded for 
three nonverbal behaviors, head nods, gaze direction changes, and 
gestures. Descriptive statistics, including reliability and standard 
error of measurement for the proficiency tests were calculated for 
the two tests and three nonverbal behavi ors in role pi ays 2 and 3 . A 
significant difference was found for head nods in role play 4. A set 
of scales for assessing nonverbal ability was constructed on the 
basis of the quantitative and qualitative analysis of the nonverbal 
behaviors in the role plays. Results of the study indicate that it is 
possible to define degrees of nonverbal abil ity. (Author/ JL) 
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Abstract 

This study attempts to design an instrument for assessing the 
nonverbal ability of foreign language learners. The subjects are 
educated Japanese non-native speakers of English (n= i 28) and 
educated North American native speakers of English (n=20) for 
baseline data. Materials used were the institutional TOEFL, for 
linguistic proficiency, the SPEAK test for oral proficiency, and a 
series of four role plays for the collection of nonverbal data. 
Proficiency tests were administered to NNS subjects who were paired 
according to the results of the TOEFL. NNS and NS subjects 
performed the three role plays in pairs and the fourth with the 
researcher's NS assistants. Role plays were transcribed and coded 
for three nonverbal behaviors, head nods, gaze direction changes, 
and gestures. Descriptive statistics, including -reliability and 
standard error of measurement for the proficiency tests, were 
calculated for the two tests and three nonverbal behaviors in role 
plays 2-3. A MANOVA was performed for the main effect Group for 
the three nonverbal behaviors in three role plays. A significant 
difference was found for head nods in role play 4. A set of scales 
for assessing nonverbal ability was constructed on the basis of the 
quantitative and qualitative analysis of the nonverbal behaviors in 
the role plays. The results of this study indicate that it is 
possible to define degrees of nonverbal ability. 
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Assessing Nonverbal Ability 

Assessing the Nonverbal Ability 
of Foreign Language Learners 

The accurate assessment of oral proficiency in a foreign 
language can be crucial for non-native speakers of English whc need 
favorable evaluations for general academic purposes at American 
universities/ for employment, to work as teaching assistants at 
American universities, or for other purposes which require a 
specified level of English proficiency. A variety of scales and 
testing met nods such as the Foreign Service Institute (FSI) 
interview and its derivative ACTFL/ETS guidelines have been created 
to assess oral proficiency but these have also been criticized for 
their narrowness as norm-referenced tests (Bachman, 199C) or fox- 
being no more than discrete-point tests without a basis in theory 
or research (Bachman, 1988; Bachman & Savignon, 1986) . 

With the expansion of our framework for communicative 
competence (Canale & Swain, 1980; Canale, 1983), there has been a 
need for improving oral proficiency evaluation using the 
communicative paradigm. Along these lines, Bachman and Palmer 
developed an oral interview test (Bachman & Palmer, 1983) in an 
attempt to account for mora of the reliable variance in the 
learner 1 s oral proficiency than existing measures (Bachman & 
Palmer, 1982) in terms of what Bachman (1988, ly90) refers to as 
Communicative Language Ability (CLA) . 

However, even Bachman and Palmer's test relies solely on the 
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vocal channel for its evaluation of oral proficiency in spite of 
recent calls to pay more attention to nonverbal behavior as an 
integral part of speech (von Raf f ler-Engel, i960; Baird, 1983) and 
clear indications that nonverbal behaviors are part of the 
scciolinguistic and strategic competence components of 
commuiiicatj.ve competence {Canale, 1983; Canale & Swain, 1930) . 

In general, language testing experts have neglected nonverbal 
behavior. Bachman's (1990) major work on language testing does not 
include nonverbal behavior under sociolinguist ic competence and it 
notes that strategic competence definitions 

include non-verbal manifestations of strategic 
competence, which are clearly an important part of 
strategic competence in communication, but which 
will not be dealt with in this book. (p. 100) 

Problems which may be related to intercultural differences in 
the use of nonverbal behaviors can, however, affect the rating of a 
second language learner's oral language proficiency causing them to 
be rated either higher or lower than their actual language 
proficiency level (Nambiar and Goon, 1993; Neu, 1990) . As Baird 
(1983) puts it, 

...the importance of nonverbal behavior in overall 
communication effectiveness is obvious, and the 
difficulties in assessing the skills involved 
should not blind us to their significance, (p. 33) 
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Nonverbal communication (NVC) is commonly considered 
"communication effected by means other than words" (Knapp & Hall, 
1992) . It has also been defined as the influencing of another 
person by means of facial expression, tone of voice, gaze, 
gestures, posture, bodily contact, spacial behavior, clothes or 
appearance/ nonverbal vocalizations, or smell (Aryyle, 1988). 
These nonverbal behaviors are also often interpreted through verbal 
signs, an indication of the importance of viewing verbal and 
nonverbal communication as opposite ends of a cont inuuiu with some 
overlap between the two (Knapp & Hall, 1 988). Thus, while they may 
be independent at the extremes of the continuum, there are 
instances where they are meaningfully integrated. 

It is useful, then, to consider the role of NVC in terms of 
this relationship with spoken language. Nonverbal vocalizations, 
gestures, facial expressions, and gaze are integrated with words 
and can "amplify and disambiguate them" (Argyle, 1988, p. 106). 
Until as recently as ten years ago, however, there were no studies 
which dealt v/ith the interactional nature of NVC (Kendon, 1984) . 

Gestures, foe example, serve a number of roles in relation to 
speech, replacing it or alternating with it in a variety of ways 
{Kendon, 1984). Nods and gaze behavior serve to regulate 
interactions as nonverbal backchanneling signals or turn-taking 
signals. Nonverbal communication also plays a part in making up 
for the inability to communicate something verbally due to 
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linguistic deficiencies, as in strategic competence (Canale, 1983) . 

While NVC can be described separately from spoken language and 
often has specific meaning independent of language (Ekman, 1976; , 
it is the integration of language with nonverbal behavior that is 
most important for assessing language learners' proficiency. 
Keiidun (1984) and McNeill (1965) both nute that nonverbal behavior 
such as gestures shares the planning process with speech and thus 
is not really completely nonverbal. In other words, rather than 
being a separate system of communication, NVC is viewed here as an 
integral part of verbal communication. 

The appropriate use of some nonverbal behaviors (in 
particular gestures, head nods, and gaze, which ameliorate the 
interaction process as backchanneling or turn-taking signals) can 
also be included under textual competence in Bachman's (1990) CLA 
model as "ways in which interlocutors organize and perform the 
turns in conversational discourse..." (p. 88). Edmondson (1981) 
also notes the importance of NVC in discourse in his observation 
that eye contact, for example, may be more important than speech 
for successful conversational development. The use of nonverbal 
behavior may even provide a key to the learner's proficiency level 
since, as Fasrch, Haastrup, and Phillipson (1984) point out, 

...the activation of strategic competence presupposes 

an inability to make use of parts of linguistic or 

pragmatic competence, (p. 168) 
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The level of use cf N7C for strategic/ compensatory purposes may 
provide a clue to the learner's level of proficiency with greater 
use indicating lower or inadequate linguistic proficiency. 

Appropriate use of NVC will also include such issues as its 
synchrony with the speaker's own speech and with that of the 
int.er iocutoi , which cciri be seen in leims of Bialystok's control 
dimension (Bialystok, 1990; Bialystok & Sharwood Smith, 1935) . It 
is expected that lower oral proficiency learners wi 11 have poorer 
synchrony or coordination due to their lower ability, affecting the 
coordination of the verbal and nonverbal planning mentioned above 
(Kendon, 1984/ McNeill, 1985) . As Faerch and Kasper (1984) note, 
...advanced learners, who are capable of planning 

longer units, can often predict a communicative 

problem well in advance and attempt to solve it 

beforehand, as part of the normal planning 

process, (p. 60-61) 
Any planning deficiency at lower levels then could also be 
reflected in the learner's use of nonverbal behaviors. 
Frequent pauses accompanied by diverted gaze by a language 
learner could be interpreted as a sign of lower proficiency 
when the actual content of the discourse is such that it 
would not place an excessive burden on the average learner's 
oral proficiency. 

The testing of nonverbal ability in conjunction with 
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linguistic and oral proficiency should concentrate on 
nonverbal features that are closely inteqrated with speech. 
The Communication Competency Assessment Instrument (CCAI) 
which was developed to assess the basic communication skills 
necessary for success at an American college (reported in 
Rubin, 1982) includes "use of nonveiba. signs appropriate 
for the situation" (p. 21) as one of 19 competencies needed 
in an educational context. Developed as. an alternative 
method for assessing academic skills, the CCAI included 
assessment of the ability to recognize and use appropriate 
gestures, eye contact, and facial expressions in academic 
situations . 

Generally speaking, any scale for evaluating the nonverbal 
benavior of speakers of English as a foreign language should also 
consider gestures (including head movement) and gaze direction. 
Such a scale should therefore include some of the most important 
functions of NVC such as backchanneling, which is often used 
differently by native English speakers and people from other 
cultures (Maynard, 1987; Maynard, 1990; White, 1989) . Japanese, 
for example, use head movement as a nonverbal backchannel signal 
more often and differently from Americans when speaking Japanese as 
well as in intercul tural discourse in English (Maynard, 1990) . 

Nonverbal behavior can also be described in terms of textual 
use, which involves the appropriate or acceptable use of nonverbal 
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behaviors to manage turns in conversational discourse, and 
strategic use, which involves compensatory behavior to make up for 
deficiencies in linguistic proficiency or otherwise clarify verbal 
messages. The three types of nonverbal behavior under the two 
categories of textual and strategic use can be used to assess 
nonverbal proficiency in conjunction with tests of oral 
proficiency : 
TEXTUAL USE 
Gestures 

• Hands are used by the speaker to emphasize speech. 

• Vertical head movement (nod) is used as a backchannel signal 
by the listener to indicate attention, understanding, or 
agreement . 

• Vertical head movement (nod) is used by the speaker as a 
wi thin-turn or turn-end signal. 

• Horizontal head movement (shake) is used by the listener to 
indicate disagreement or to accompany laughter. 

Gaze 

Listener-directed gaze is used at the end of an utterance to 
elicit a backchannel response. 

• Terminal gaze (prolonged gaze starting just before the end of 
an utterance) is used to signal the end of the utterance. 

• Speaker-directed gaze is used to signal attentiveness . 
Facial Expressions 
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Smiles are used to indicate attention or agreement. 

Frowns are used to indicate disagreement or lack of 

una* (.standing. 
STRATEGIC USE 
Gestures 

Mime (hand gestures) is used to compensate for a linguistic 
deficiency such a?; the lack of a necessary lexical item. 
• Hand gestures are used to support spoken language to 

communicate spacial relationships and physical shapes which 
are not always easily understood using spoken language alone. 
This list of nonverbal behaviors is limited to hand gestures, 
head movement (nods), gaze direction in relation to the 
interlocutor, and facial expressions as behaviors which can be 
easily observed and rated by a language teacher. It is not, 
therefore, meant to be a complete list of nonverbal behaviors, but 
rather a short list of behaviors in which differences may be 
readily observed and may vary as a result of differences in the 
oral proficiency level or cultural background of the learner. 
Furthermore, while such a list gives the impression that these are 
discrete behaviors, there is a degree of integration among the 
nonverbal behaviors themselves. Nonverbal backchanneling, for 
example, may include a nod at the listener accompanied by 
listener-directed gaze. This list is meant to simplify and focus 
the rating task by grouping behaviors according to their role in 
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The purpose of this study, then, is to develop scales for 
assessing foreign language learners 1 nonverbal ability as a 
component of their era] proficiency. In order to develop a method 
for assessing nonverbal ability in a English as a foreign language, 
the following research questions will be addressed. 

1. What nonverbal behaviors (such as gestures, gaze, and facial 
expressions) do native speakers (NS) and non-native speakers 
(NNS ) use? 

?. How do NNS nonverbal behaviors differ from those of NSs in 
similar situations? 

3. If NNSs do exhibit different nonverbal behaviors from NSs, to 
what extent is this related to oral L2 proficiency? 

4. To what extent is it possible to identify degrees of 
nonverbal proficiency? 

Method 

Subjects 

The subjects in this study are 28 non-native speakers and 20 
native speakers of English. The non-native speakers are educated 
middle-class Japanese male and female EFL learners consisting of 24 
students, two faculty members, and. two office staff members of a 
Japanese university. The native speakers are 20 educated white 
middle-class North American males and females who are EFL teachers 
or graduate students in TESOL at a Japanese branch of an American 
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university in Japan. They provide the baseline data for nonverbal 
behaviors , 

The Japanese subjects were recruited using a poster placed on 
a university bulletin board and also directly from among the 
researcher 1 s students, the office staff, and tne faculty in 
exchange foi a Iree administration of the TOEFL. While a ranuom 
sample would be desirable, this sample is ecologically valid since 
those who are required to take oral proficiency tests generally 
self-select into the programs or situations which require the 
tests. While such a sample presents potential problems for 
statistical analysis, the subjects do reflect the type of person 
who would actually be tested in this situation. 
Jnstruments 

Instruments used in this study are the Test of English as a 
Foreign Language (TOEFL) as a general English proficiency measure, 
the Test of Spoken English (TSF.) as a measure of oral proficiency, 
and a series of role plays developed by this researcher to elicit 
data for the development of nonverbal ability (NOVA) scales. 



The Institutional Testing Program (ITP) version of the Test 
of English as a Foreign Language was used to assess overall English 
proficiency because of its generally high reliability of r=.95 and 
validity. The TOEFL meets the criteria of content validity for the 
university setting of this study with its academic content, and as 
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a test ct linguistic competence it is appropriate for ins construct 
validity. Its hiqh correlations with other measures of English 
proficiency (ETS, 1992) gives it a high degree of criterion-related 
validi ty . 
TSE 

Oral proficiency was measured with Test Form 1 of the 
Speaking Proficiency English Assessment Kit (SPEAK) version of the 
Test of Spoken English (TSE) , a "semidirect" measure of oral 
proficiency which rates pronunciation, grammar, fluency, and 
overall comprehens i bi 1 i ty . Tnter rater reliability of the TSE ha? 
been shown to range from .87 to .92 for the four categories of 
ratings (ETS, 1990) . Content validity is met since the TSE was 
developed so that the items chosen would reflect: performance on 
similar items on the TOE Fh and Foreign Service Institute (FS1) oral 
proficiency interview. Criterion-related validity is supported by 
studies which have shown it to correlate well with the FSI rating 
of oral proficiency at .79 (ETS, 1990) . 
Role Plavs 

Nonverbal data was elicited using four role plays performed 
in English (two repeated in Japanese as explained below in 
Procedures) . The first role play between two friends was used as a 
warm-up and to reduce the reactivity effect of the video cameras. 
Since some subjects had never performed role plays before, this 
informal format was easier for them to begin with. The following 
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two roie plays were between unequals in an interview format (a 
language school interview and a part-time iob interview) since many 
oral proficiency tests have an interview format in which the 
interviewer is superior to the person interviewed- This NNS/KNS 
format also made it possible to compare these interactions with 
NS/NS inter actions as well as NS/NWS interactions. In the fourth 
role play, the subject was required to make a request of a teacher 
for a letter of recommendation (see Appendix A for English versions 
of the role plays) . 

The role plays were in a format si mi ] ar to the simulations 
used by Haastrup (1986) as part of what is called a "structured 
oral incerview" (p. 76) for assessing oral proficiency. In this 
simulation, pupils were given an instruction card telling them that 
they were cpplying for a job and have been called for an interview. 
The cards also indicated in the pupils 1 native language what 
information would be required. This kind of test is widely used in 
Denmark to test English language students after five years of 
English instruction. 

The role play format has content validity as a format which 
is used in oral proficiency testing (Bachman & Palmer, 1983; 
Haastrup, 1986) and to elicit data for discourse analysis 
(Edmondson, 1981; Kasper 1981), and it is expected to elicit the 
most natural NVC that can be expected in a testing situation. 

Video recordings of the subjects performing the role plays in 



Assessing Nonverbal Ability 

1 5 



English and Japanese (for Japanese subjects) were used tc analyze 
nonverbal behavior and to provide the basis for the development of 
the scale for measuring nonverbal ability. 
Procedures 

The English versions of the role plays in the Apoendix A were 
piloted with native speakers and Japanese university students. As 
a result cf the initial piloting, it was decided to have the 
subjects sit facing each other with a table between them and to 
take the role play cards away from the subjects before beginning 
because frequent referral to them inhibited the naturalness of the 
conversations as well as the nonverbal behavior itself. The 
contents were then revised and translated into Japanese so that the 
subjects 1 understanding of the role plays would not depend on their 
English proficiency and so that the language they used would come 
from their own English ability and not from the cards. This would 
also insure that the strategic use of NVC would be elicited for 
unknown vocabulary. Subjects performing the pilot role plays, for 
example, did not know how to say ,; letter of recommendation" for 
role play 4. Almost all of them mimed the shape of a piece of 
paper while explaining what they needed. 

The Japanese translations were checked by a Japanese English 
teacher and piloted with another pair of Japanese university 
students. In this piloting, the subjects repeated the two 
interview role plays in Japanese after performing the final English 
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rcle play individually with a native speaker, This piloting showed 
that subiects could easily perform the role plays in the allotted 
time and would not need another warm-up role play in Japanese 
before the Japanese performances. 

The institutional TOEFL was administered to the subjects 
together in a classroom at their university by an experienced TOEFL 
supervisor. The SPEAK Form 1 version of the TSE was administered 
individually to each subject in the researcher's office according 
to the instructions supplied with the test. The researcher 
remained in the room to observe them and assist if there were any 
problems. The recordings of these tests were rated by two 
experienced native speaker EFL teachers according to the TSE 
guidel ines . 

The subjects were paired according to their TOEFL scores to 
insure that there would be as equal a linguistic burden as possible 
between subjects when performing the role plays. They performed the 
role plays in the researcher's office, where they also took the 
TSE, seated facing each other with a small table betv:een them. 
Their performances were recorded using two video camcorders placed 
unobtrusively pointed at less than 45 degree angles toward each 
subject. A.fter the subjects were seated, they were given the 
Japanese versions of the role play instructions shown in Appendix 
A. ' After signifying that they understood, they were given the role 
play cards for role play 1 (warm-up role play) between two friends. 
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The subjects were given four minutes to perform this and subsequent 
role plays and did have access to the cards during the 
performances . 

After performing role play 2 (a language school interview) 
and role play 3 (a part-time job interview), which were designed to 
elicit equivalent data from both subjects by giving each a chance 
to play the status-higher role of the interviewer, each subject 
performed role play 4 (asking for a letter of recommendation) with 
the researcher's native speaker assistant who was an experienced 
F,FJ. teacher. Finally, the subjects replayed role plays 2 and 3 in 
Japanese to enable the comparison for the subsequent study of their 
nonverbal be.iavior when speaking Japanese with their performances 
when speaking English. As previously stated, there was no warm-up 
for the Japanese performances. 

Of the 15 NNS pairs originally performing the role plays, one 
subject was not included in the analysis because he was Chinese and 
another was excluded because his glasses prevented a clear view of 
his gaze direction. Their partners were included in the 28 
subjects analyzed here. 

The English versions of the role plays were performed as 
above with the 20 North American NS subjects to obtain baseline 
data on nonverbal behavior by NSs in similar situations. These 
were performed at facilities of their university and recorded using 
two video camcorders placed as with the NNSs. Another American NS 
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assistant performed role play 4 with each of the subjects. 
Additional instructions for the NS assistants are included in 
Appendix A. 

The role plays were then transcribed using the discourse 
transcription (DT) method of Du Bois, Schuet ze-Coburn, Cumming, and 
Paolino (1933). Based on the review of the nonverbal behaviors 
made during this transcription, nods, gaze direction, and gestures 
were chosen for analysis and coded according to a simple 
transcription system developed by the author and shown in Appendix 
B. Only the middle three minutes of each role play were coded for 
analysis to eliminate greetings and formulaic speech at the 
beginning and to assure that none of the pairs finished too early. 

In this system, nods were coded as nodding occasions, meaning 
the performance of one or more nods as one nonverbal signal. 
Nodding occasions included eight of Maynard's (1989) 1C contextual 
categories of vertical head movement: as a backchannel 
continuer/pause filler with or without verbalization, as a 
continuer signal by the listener to the speaker, as a clause 
boundary or turn-end signal occurring simultaneously with speech, 
as a turn transition claimer or transition filler during pause 
only, with expressions of affirmation or agreement such as "yeah" 
or "uhuh, " for emphasis, and as a preturn claim. Rhythmic nodding 
with speech was not counted. 

Gaze direction was labeled as "toward" when the subject was 
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perceived to change gaze direction toward the interlocutor. It was 
labeled as "away" when when the subject was perceived to change 
gaze direction away from the interlocutor. Head direction was also 
used as a measure of gaze direction (Duncan & Fiske, 1985) . For 
the purpose of analysis, only the number of changes in gaze 
direction toward the interlocutor were counted. 

Gestures labeled included miming to compensate for linguistic 
deficiencies and gestures to support or enhance speech by 
indicating direction or location, spaciai relationships, counting, 
shapes, and size. Gesturing for emphasis was not included. A 
brief description of the movement and meaning of the gesture was 
included in parentheses after the code for gestures. 

Analyses 

The dependent variable in this study is nonverbal behavior. 
The independent variables are linguistic proficiency measured by 
the TOEFL and ora] proficiency measured by the TSE. The control 
variable is nationality with only Japanese subjects used for rating 
nonverbal ability. Potential moderator variables are sex, age, and 
years of English study. 
Nonverbal Data 

Nonverbal data was analyzed by observing the role play videos 
of the NNS subjects and NS subjects and noting similarities and 
differences among the NNS subjects 1 in the performance of the three 
nonverbal behaviors listed and comparing NNS and NS performances in 
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English. The three types of nonverbal behaviors which show 
potential for ratinq nonverbal ability were counted and a 
multivariate analysis of variance (MAN OVA) was carried out to 
determine if there was a significant difference between the mean 
number of occasions for the NNS and NS subjects' nonverbal 
behaviors in each of Lhe role plays. The results fur role plays 2 
and 3 were averaged for th» purpose of analysis resulting in three 
nonverbal scores for each type of interaction: NNS /NNS and NNS/NS 
assistant for Japanese subjects and NS/NS and NS/NS assistant for 
baseline data subjects. This averaging was carried out to 
compensate for differences which may have been a result of the 
alternating roles in role plays 2 and 3. Although gaze direction 
while listening has been found to be consistent among members of 
the same culture in spite of individual differences (La France & 
Mayo, 1978), it may bo affected by the speaker's identity (Kendon & 
Cook, 1969), and the amount of listening itself is affected by 
whether the higher- or the lower-status role is being played. The 
role play results for the Japanese language interactions are not 
included in this study. 
Statistical Analysis 

Data consist of interval scores from the TOEFL, the TSE 
ratings of pronunciation, grammar, fluency, and overall 
comprehensibility, and the counts of nonverbal behaviors in role 
plays, as well as reliability coefficients for proficiency tests. 
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Reliability 

Reliability is an important measure for assuring the 
consistency of testing instruments. Theoretically, high 
reliability means that test takers can be expected to receive 
similar scores on repeated administrations of the test. Lower 
reliability means a greater range of difference in possible scores 
for such repeatea administrations. Reliability for the TOEFL for 
these subjects was calculated using Cronbach's alpha coefficient 
for internal consistency. lnterrater reliability for the TSE was 
calculated using Pearson correlation corrected with the 
Spearman-Brown prophecy formula used as suggested by Brown (1992) 
for decision reliability. 
Validity 

While reliability is an important quality for a good test, it 
is of no use without validity, which is the extent to which a test 
actually measures what it claims to be measuring. As previously 
stated, the validity of the TOEFL and TSE are well established. 
They appear to measure what they claim to be measuring and 
correlate well with other measures of the same constructs, 
establishing criterion-related reliability . 

Nonverbal data collection is based on descriptions of 
nonverbal behaviors found in the literature and supported by 
baseline data gathered from native speaker subjects, thus 
demonstrating content validity. 
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Other stat i sties 

Other statistics will include descriptive statistics (mean, 
range, standard deviation, and standard error of measurement) for 
the TSE ratings and TOEFL scores. 



Table 1 shows the descriptive statistics for the TOKfc'L and 
SPEAK test. As expected, the results of these tests arc generally 
normally distributed and reliable measures of proficiency with the 
exception of the SPEAK grammar section which includes only two 
scores for grammar, making it a poor measure for lower proficiency 
subjects who may score zero on one or both parts of this section. 



Table 2 shows the descriptive statistics for the number of 
occasions of head nodding, gaze toward the interlocutor, and 
meaningful gestures in role plays. The distributions are generally 
normal, with the exception of gesture distributions which is 
positively skewed. At least one subject in each type of 
interaction performed a number of gestures that was well above the 
mean. Figure i shows the mean number of occasions graphically. 
There is a clear difference in the means for all behaviors in each 
role play with a consistently greater number of NNS nods in NNS/NNS 



Results 



Insert Tables 1 and 2 



and Figure 1 about here 
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and N T NS/XS interactions and a greater number of gaze direction 
changes toward the interlocutor and gestures in NS/NS interactions. 

Since the KAN OVA for the Group main effect was found to be 
significant, F(6, Al) = 1.412 for the Pillais, Hoteliings, and 
Wilks 1 lambda tests of significance at p < .0001, univariate ANOVAs 
were explored for each nonverbal behavior in each type of 
interaction. The effect for Group was significant (F=27.G04; 
df=1/46; p<.0001 ) for head nodding in rol* play 4, a finding 
consistent with Maynard's (1987, 1990). 

Discussion 

Several of the findings here provide possible insights which 
contribute to the preliminary drafting of a set of scales for 
assessing nonverbal ability. There is, however, a significant 
difference between a nonverbal rating scale and one which 
concentrates on oral proficiency. In the case of oral proficiency, 
the rating will concentrate on production, as in the case of the 
SPEAK test used here, while leaving the measurement of receptive 
skills to written tests such as the TOEFL. Nonverbal 
communication, on the other hand, is unique in that it covers both 
reception and production in a given interaction and it is not 
always accompanied by speech. 

As for what behaviors NNSs and NSs use in similar situations, 
this study found through quantitative and qualitative analysis that 
vertical head nodding, gaze direction, and hand gestures would be 
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the most reveal inq behaviors because they are easy to observe and 
interpret and because they occur frequently even in a limited 
three-minute period for both NS and NNS interactions. 

In addition, NNS behaviors are different from these of NS 
subjects in this study. The number of NNS nods, as to be 
expected, were significantly yreatex. in KNS/KS interactions and 
slightly greater even for NNS/NNS interaction, though not 
significant. What makes this a very ratable behavior is not only 
the frequency but also the nature of the occasions in which they 
occur. This cn 1 tura 1 1 y- based difference can be seen in differences 
in backchanneling (Maynard, 1937, 1989, 1990: White, 1989), as 
f ol lows : 

(1) (Role Play 4 - Request for Recommendation, NS4A) 1 

Professor: How long will it take you? 
Student: urn, 



A 



. . I figure, 



two vears. 



T 



Professor : 



Probably . 



(Role Play 



4 - Request for Recommendation, NNS13A) 



Professor : 



these are all important things. 



Student : 



Uhuh. 



N 



Pro f essor : 



uh, 



Kaybe you could write down just, 
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your basic, 
A 

uh, 

your schooling, 
N 

and, 

(3: (Role Play 4 - Request for Recommendation, NNS12B5 

Professor : when do you want me, 

to write the letter. 

A 

. . . next week, 
T W 

or, 

next [month] 

Student : [ ah] , 

Yeah. 

urn, 
A 

. . .Next month, 
T N 

[and] , 
A 

Professor: [By next] month. 

The NS in example (1) had approximately the average number 
of head nods for baseline data subjects. This subject had no 
examples of backchannel head nodding. Most nodding was for 
emphasis or as a turn- end signal/ as in this example. The NNS 
subject in example (2) was a high oral } roficiency subject with a 
less than the average number of head nods. This subject had 
examples of turn-end nods, and nods with expressions of affirmation 
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cr agreement such as "yeah" or "uhuh, " which occur frequently in 
NNS data. The NNS subject in exair.ple (3) was a low oral 
proficiency subject who had slightly more than the average amount 
of head nodding for NNS subjects. There was frequent nodding as a 
backchannel/continuer signal without verbalization while listening 
dij well as with affirmation ui ay l eemeu t . Of course, the oral data 
itself did differ between proficient and less proficient subjects. 
NS and higher oral proficiency subjects spent less time listening 
and more time talking in role play A than lower oral proficiency 
NNS p because they understood what the professor was asking them. 

As for gaze direction, while there are similarities between 
NS and NNS subjects for frequency of changes and certain types of 
usage, there are greater differences between higher and lower oral 
proficiency subjects. Examples (1) and (3) above illustrate how 
both NS and NNS speakers precede the answer tc a question with a 
question with a filled pause, "urn," accompanied by a gaze change 
away. Usually the next gaze change toward comes at or near the 
beginning of the answer. Lower oral proficiency subjects often 
delay their gaze change toward the interlocutor, as follows: 
(4) (Role Play 4 - Request for Recommendation, NNS15B) 

Professor: Do you have any hobbies. 

Did you belong to a club. 

A 

Student: J , 

I belong to, 
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Professor: 



. . . table tennis club. 

T N 

Vhuh. 



N 



Mhm. 



In America they call table tennis ping pong. 



This lower proficiency subject looks away at the end of the 
rephrased question by the researcher's NS assistant and does not 
change direction toward the interlocutor until the end of the 
answer. This could be related to control variability (Bialystok & 
Sharwood Smith, 1985), affecting the synchrony of gaze direction 
and speech due to the learner ! s uncertainty over what form of 
languaoe is appropriate. Gaze away is usually a sign of thinking, 
therefore prolonged gaze away may be sign of the learner's lower 
proficiency, and frequent and long occasions of gaze away could be 
rated as a sign of "lower" nonverbal ability. 

Differences between NSs and MNSs for hand gesturing to 
enhance speech or to compensate for linguistic deficiency cannot be 
interpreted only on the basis of simple counting and comparisons of 
means. Content and success are also important factors. 

NSs did not necessarily use hand gestures. Two out of 20 
baseline data subjects did not gesture in any of the role plays as 
compared to three out of 28 NNS subjects. The two major 
differences between NSs and NNSs, aside from compensatory usage by 
NSs, w a s the variety and type of gestures used. Frequency itself 
is not an indicator. One NNS used hand gestures pointing at his 
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own chest nine times to indicate "1, 11 "my, 11 or "me" out of a total 
of 11 hand gesturinq occasions. The other gestures were pointing 
at the NS assistant for "you" and holding up four fingers while 
saying "four years," While NSs sometimes pointed at their chest 
for "I," the NNSs overuse could be considered inappropriate by 
native speaker norms. Inter estinyly, although it was expected that 
some of the NNSs would point at their noses for. "me" as is often 
encountered in Japan, none of these subjects did. 

One lower proficiency subject used hand gestures 13 times, 
four times for "me" and twice for "you." The resr of the gestures 
were for direction or miming the shape of a "letter of 
recommendation" or the act of writing. It is interesting that the 
researcher had earlier thought that this subject was more 
proficient, until very low scores on the TOEFL and SPEAK test 
suggested otherwise. The subject's use of gaze direction and head 
nodding also seemed to be more appropriate by native speaker 
standards . 

Success or failure at an attempt to use mime as a 
compensatory strategy can also be seen as an indicator of the 
learner's level of nonverbal ability, as follows: 
(5) (Role Play 3 - Convenience Store, NNS6A) 



Student : 



urn, 



. . .What situation. 
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Manaoer : 



urn, 
A 

...<6)<L2 nan to iu L2> ((What do you say?)) 



Student : 



G(mimes carrying something with both hands) 
Ca- carry. 



Manager : 



Carry. 
T N 



During a long pause, the subject playing the manager cannot think 
of a lexical item. Before reverting to code switching, she begins 
to mime the word that she cannot think of. The subject accepts the 
interlocutor's interpretation, indicating that the gesture for 
compensatory purposes was successful. If the interlocutor had not 
supplied an acceptable word, this example would be considered a 
failure . 

Thus, while the performance of some kinds of nonverbal 
behavior are related to language proficiency, there are exceptions 
to this, as Neu (1990) has pointed out. This supports the idea 
that nonverbal ability can be treated as a separate construct from 
linguistic proficiency. For testing purposes, nonverbal ability 
can be integrated into Bachman's (1990) CLA model as shown in 
Figure 2. 



As a result of the quantitative and qualitative analysis 
carried out in this study, a srr of scales for assessing nonverbal 



Insert Figure 2 about here 
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ability can be constructed 



as shewn in Table 3. 



Insert Table 3 about here 



Under textual ability, looking away and infrequent nodding 
are associated with lower textual ability for frequency, and 
backchanneiing frequently and at every pause such as Japanese 
speakers do would be associated with lower textual appropriateness. 
Jnder strategic ability, compensatory usage will be rated according 
to need to use and successful performance, wh;le the rating for 
appropriateness would depend on how often gestures are used and the 
content of the gesturing. The subject who gestured for "me" 
frequently and occasionally pointed at his or her nose would be 
rated lower for appropriateness. 



This study has investigated the possibility of assessing the 
nonverbal ability of foreign language learners. It has 
demonstrated that: 

1. Vertical head nods, gaze direction changes, and gestures are 
nonverbal behaviors that are used meaningfully by both NSs 
and NNSs in simi situations. 

2. These nonverbal behaviors differ in frequency, in their 
synchrony with speech, and in content between NSs and NNSs. 

3. The use of these nonverbal behaviors by NNSs can differ 



Conclusion 
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according to oral L2 proficiency. 
4. Deqrees of nonverbal ability can be identified and applied to 
a meaningful rating system. 

As a pilot study, there are limitations to the scales 
outlined here. Only three behaviors are applied here. This study 
did not include horizontal head movement (shake), facial 
expression, posture, or interpersonal distance. The three 
behaviors chosen, however, can be easily observed by raters. These 
scales are meant for research purposes, therefore only one cultural 
group was used as a control for data roileol'ion. Much work remains 
to be done before this can be applied outside of research, 
especially in the area of norms. 

The next step is to test the test. By applying these scales 
to the NS/NNS data in this study using independent raters, it will 
be possible to answer the questions of whether the scales arc 
reliable and whether nonverbal ability is an independent construct 
from linguistic and oral L2 proficiency. 
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Appendix A - English Versions of Role Plays 

Role Play Instructions 

You will be asked to pertorm a number of roJe plays. 
Read each situation and imagine that you are really the 
person in the role play. Specific details are not 
given/ but try to speak in as much as detail as 
possible . 

Role Play 1 (Warm-up) 

Role A: You are at the school cafeteria talking to a friend who 

is very good at Japanese. You want to borrow some 
books from a friend so that you can study Japanese. 

Role B : You are at the school cafeteria talking to a friend who 

wants to study Japanese. You are very good at Japanese 
so your friend wants to borrow some books from you to 
study. Offer your friend some books and try to give 
some advice for studying Japanese. 

Role Play 2 

Role A: You are a student applying to enter an English language 

school. You are being interviewed by one of the 
teachers. Answer the teacher's questions about 
yourself, your interests, and your reasons for studying 
English in as much detail as possible. 

Role B: You are a teacher at an English language school. You 

must interview an applicant to evaluate his speaking 
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ability. First, ask the applicant basic background 
questions (name, birthplace, etc.). Then, ask about 
interests, language learning experience, and reasons 
for studying English. Ask in as much detail as 
possible . 

Role Play 3 

Role A: You are the manager of a convenience store. You 
must interview a student for a part-time job. 
First, ask the applicant basic background 
questions (name, hi rt hp] ace, etc, ) . Then, ask 
about interests, working experience, and reasons 
for wanting to work at your store. Ask in as 
much detail as possible. 

Role B: You are student applying for a part-time job at a 
convenience store. You are being interviewed by 
the manager of the store. Answer the manager's 
questions about yourself, your interests, and 
your reasons for wanting a job at the store. 
Answer in as much detail as possible 
(Role Play 4 is performed individually by both subjects with a 

native speaker) 
Role Play 4 - Non-native Speaker 

Role A/B: You want to apply to an American university's graduate 
school to study in your major. You need a letter of 
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recommendation from your teacher. You go to your 
teacher's of : ice to ask him for one. Be ready to 
explain why you need the letter, what information it 
should contain, when you need it, and what your teacher 
should do with it in as much detail as possible. 
Role Play 4 - Native Speaker 

Role B/A: You are a professor at a Japanese university. A 
student comes to you to ask for a letter of 
recommendation so that he can study at an American 
University. Ask the student why he wants t^ ritudy 
abroad, what information the letter should contain, 
when he needs it, and where he should send it. Try to 
get the student to explain in as much detail ?.s 
possible . 

Instructions for NS Assistant 

1. Greet the student: "Hi, (name), what con I do for you?" 

2. Remember. You do not know what the student wants beforehand. 

3. Help the student speak, but do not put words in his/her 
mouth. Th i burden to communicate should be on the student as 
long as pauses are not too long. Be patient. 

4. If the student does not give instructions about the letter of 
recommendation, ask questions about why he/she wants to study 
abroad, the information to be contained in the letter, when 
it is needed, where to send it, and so on. 
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5. You will hav. four minutes for the role play, but do not 
worry about the time. If there is tine, close the 
conversation with small talk and a word of "good luck." 
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Appendix B - Transcription Symbols 



Di scour so TrariScription (Du Bo.is 
Uni ts 

Intonation unit 
Truncated intonation 

unit 
Word 

Truncated word 
Speaker identity/ 

turn start 
Speech overlap 
Transitional Continuity 
Final 

Cont a nuing 
Appeal 

Pause 

Long 
Medium 
Short 
Latching 
Vocal Noises 

Vocal noises 



a), 



1993) 



Laughter 

Tra scr iber s Perspect ive 

Researcher T s comment 
Uncertain hearing 
Indecipherable syllable 

Specialized notation 

Nontranscription line 



{ carriage return} 



! space ) 



(N) 



(0) 
( ) 

THROAT , GULP , SWALLOW , 
SNIFF, COUGH, TSK (tongue 
click) 
0 

( ( ) ) 

<x x> 

X 
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Code switching 

Margir.a 1 words 

hesitation 
backchannel 
arfirmation (be) 

negative be 
Nonverbal Transcription 
Role identity 

Nonverbal notati on 

Hand gestures: 
Head Movement 

Head nods: 

Gaze 

Gaze in direction 
of interlocutor 

Gaze away from 
interlocutor 



<L2 L~> 

uh, unh, urn, er 

hm, m, huh, hunh 

mhm, unhhunh, uhuh, oh, 

ah 

unh-unh 
a ) 

b) (subject B: symbols are 
ir. italics) 

G ("brief description") 

N 

T 
A 
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Footnotes 

1 All examples are preceded by the number of the example, the 

title of the role play, and the subject number preceded by NS or 
NNS. Coding is made at the approximate onset of each occasion. 
Duration is not included. Nonverbal notes apply to the subjects 
for listening and speaking and are not included for the NS 
assistant's behavior. See Appendix B for nor. v/erbal transcription 
symbols . 



45 



Assessing Nonverbal Ability 

45 



Table 1 Descriptive statistics for proficiency tests 

TOEFL SPEAK 

PL 1 JPL 2 Pt. 3 Total . Pron. Grammar . Fluency Coinprehensibility 

N=28 

Mean 41.07 39.46 40.43 403.21 1.15 1.01 1.07 111.21 

SD 4.77 8.07 7.52 59.66 .49 .65 .44 46.30 

Low-hiah 32-52 26-57 28-60 310-553 0.3-2.2 0-2.3 0.3-2.15 30.5-210 

r„ " -93 

.94 .97 .94 .98 



SEM ~_ _~ 15,78 A2 .11 . ... „...LL ._. A 55 _ 

* Corrected with the Spearman-Brown prophecy formula 



Table 2 Descriptive statistics fo. 
Nods 

Role Play: 2 3 3/4 4 2 

Native Speakers (N = 20) 

Mean 20.550 21.40 20.975 15.200 21.45 



SD 


6.461 


11.04 


6.290 


6.202 


12.65 


Low-high 


8-35 


7-43 


7.5-33 


6-29 


4-51 


Range 


28 


37 


26.5 


24 


48 


Nonnative Speakers ( N 


= 28) 








Mean 


21.750 


23.390 


22.571 


30.89 


17.710 


SD 


8.374 


9.645 


8.525 


11.98 


6.943 


Low-High 


4-34 


5-38 


7.5-36 


14-61 


7-37 


Range 


31 


34 


28.5 


48 


31 



r nonverbal behaviors in role plays 



Gaze Gestures 

















3 


3/4 


4 


2 


3 


3/4 


4 


21.750 


21.600 


24.800 


3.550 


3.200 


3.375 


5.600 


9.767 


S.433 


7.527 


3.735 


3.458 


2.687 


4.587 


6-44 


8-39 


14-48 


0-13 


0-13 


0-8 


0-16 


39 


32 


35 


14 


14 


8 


17 


18.500 


18.107 


23.320 


2.107 


2.714 


2.411 


4.179 


5.137 


5.547 


6.403 


2.87 


2.788 


2.560 


4.660 


10-30 


8 5-33.5 


14-37 


0-9 


0-10 


0-9.5 


0-18 


21 


26 


25 


10 


11 


10.5 


19 
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M 




Native speakers 
■ Non-native speakers 



Nods 2/3 Nods 4 Gaze 2/3 Gaze 4 Gesture 2/3 Gesture 4 
Nonverbal behaviors by role play and group 



Figure 1 Nonverbal behaviors i n rol e plays 



Communicative Language Ability 



A Language competencies 

1 Grammatical competence 

2 Textual competence 



3 lllocutionary competence 

4 Sociolinguistic competence 
B Strategic competence 



C Psychophysiological mechanisms 



Nonverbal ability 
- Textual ability 



Strategic ability 



Fiqure 2 Communicative Lanquaqe Ability (CLA) (Bachman, 
1990) with a nonverbal component 
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Table 3 Scales of nonverbal ability (NOVA) 



Textual Ability 



Rating 

Frequency 

C Extremely limited use of 
head nods and infrequent 
changes in gaze direction 
toward partner in 
conversa t i on 

1 Frequent use of head nods 
and changes in gaze 
direction that are not 
acceptable by native speaker 
norms 

2 Frequency of head noas and 
changes in gaze direction 
approaches native speaker 
norms 

3 Frequency of head nods and 
changes in gaze direction 
acceptable by native speaker 
norms 



Rating 

Appropriateness 

0 Totally iriappiopj: iale use of 
head nods and gaze direction 
by native speaker norms 



1 Frequent inappropriate use 
of head nods and changes in 
craze direction 



2 Rare inappropriate use of 
head nods and changes in 
gaze direction 

3 Use of head nods and changes 
in gaze direction acceptable 
by native speaker norms 



(Table Continues) 
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Rating 

Compensatory Usage 

0 No evidence of hand gestures 
to solve considerable 
linguistic problems 

1 Limited use of hand gestures 
to solve linguistic problems 
with occasionally 

i 

unsuccessful results 

2 Hand gestures successfully 
used to solve linguistic 
problems 

3 Rare linguistic problems 
requiring the use of hand 
gestures for compensation 



Ability 
Rating 

Appropriateness 

0 Never uses hand gestures to 
support or enhance meaning 

1 Occasionally uses hand 
gestures to support or 
enhance meaning, often 
inappropriately by native 
speaker norms 

2 Use of hand gestures 
approaches native speaker 
norms 

3 Use of hand gestures 
appropriate by native 
speaker norms 
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